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ABSTRACT 

The cost-effectiveness and impact on response rate of 
variations in mailing procedures involved in a mail survey of teacher 
education graduates were ar^sessed. Variables studied were use of a 
preliminary or advance letter, postcard versus letter reminders as 
the first follow-up mailing, timing of the reminder (1 versus 2 weeks 
after the first mailing), use of a reminder versus no reminder prior 
to the mailing of the duplicate questionnaire, and timing of the 
mailing of duplicate questionnaires (3 versus 4 weeks following the 
first mailing when no reminder was sent). Subjects were 288 
individuals who had completed teacher preparation programs leading to 
initial certification at a major institution in the Southeast from 
the fal] quarter of 1987 through summer quarter of 1988; and 288 
individuals identified through comparison of applications for 
certification, student teaching records, and graduation lists. Daily 
tabulations of returns for each experimental condition were 
maintained. Cost analyses covered outgoing postage; return postage; 
mailing labels for envelopes; letterhead stationary and envelopes; 
plain return envelopes; and duplication of letters, postcards, and 
questionnaires. Results indicate that the preliminary letters are not 
cost effective, while reminders and duplicate follow-up 
questionnaires are cost effective. Three tables and one figure are 
included. (TJH) 
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Variations in Mail Survey Procedures: 
Comparison of Response Rates and Cost 

When conducting surveys, a high response rate is always desirable. In many situaUons, however, there may 
be financial limitauons that prevent the researcher from using techniques that have been shown to elicit the 
maximum number of responses. This is frequendy true when die research is not financed by external sources. The 
mediodologicai choices made by die researcher in designing die study are dius influenced by bodi die desire for a 
high response rate and die resources available. 

Results of previous studies of sur»ey mediods are often conflicting. The use of a preliminary letter or post- 
card in mail surveys has been found to produce higher response rates by many researchers (Eisingex, Janicki, 
Stevenson, & Thompson, 1974; Ford, 1967; Heaton, 1965; Stafford, 1966; Walke- & Burdick, 1977), although 
Parsons and Medford (1972) found it made no difference. The use of a foUowup has been more effective in inducing 
responses dian preliminary leUers (Kephart & Brcssler, 1958), but foUowups are not all identical. Postcaid and/or 
letter reminders have significandy increased returns in some studies (Hinrichs, 1975; Moss, 1981; Nichols & Meyer, 
1966; Wiseman, 1973) but had only a slight effect (diat was eliminated when coupled widi a second questionnaire) in 
anodier (Rossman & Astin, 1974). 

Timing of mailings, which varies widely, may be anodier variable to consider. Little research has been 
conducted on timing widiin a single study odier dian diose studies diat do or do not use a followup reminder. 
Researchers generally base dieir judgments on response paUems to more dian one independent survey. Lindsay 
(1921) compared response rates from two studies widi different timing of follnwups, while Nichols and Meyer 
(1966) used postcard reminders after diree, 16 and 27 days. Cox (1966), based on several studies, recommended 
reducing die interval between mailings to seven days. 

The objectives of die present study were to determine die cost effectiveness and impact on p x)nse rate of 
variations in mailing procedures in a mail survey of teacher education graduates. The variables stud.vu were: use of 
a preliminary or advance letter, postcard versus letter reminder as die first foUowup mailing, one week versus two 
week timing of lemuider, reminder versus no reminder prior to mailing of duplicate questionnaire; diree week versus 
four week tim Ag of mailing of duplicate questionnaire when no reminder was sent 
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Method 

Subjects 

Subjects for the survey were 288 individuals who had completed teacher preparation programs leading to 
initial cerUficaUon at a major insUtudon in the Southeast during fall quarter 1987 through summer quarter of 1988. 
In the fall of 1988, a total of 288 individuals were identified through comparison of applicaUons for certification, 
student teaching records, and graduation lists. 
Procedure 

Individuals were randomly assigned to one of two conditions: preliminary letter or no preliminary contact. 
The 144 individuals in each subgroup were randomly assigned to the following six condiUons: postcard reminder 
after one week; postcard reminder after two weeks; letter reminder after one week; letter reminder after two weeks; no 
reminder, with duDlicate quesUonnaire mailed after three weeks; no reminder with duplicate quesUonnaire mailed after 
four weeks. Twenty-four individuals were assigned to each of the 12 groups. The design of the study is shown in 
Figure 1. 



Type of First 
Reminder 


Timing 


Preliminary Letter 
Ye55 No 


None 


OuDlicate q»e<?tionnaire at 3 weeks 


PI Nl 




Duplicate questionnaire at 4 weeks 


P2 N2 


Postcard 


1 week, duplicate questionnaire at 3 weeks 


P3 m 




2 weeks, duplicate questionnaire at 4 weeks 


P4 NP4 


Letter 


i week, duplicate questionnaim at 3 weeks 


P5 N5 




2 weeks, duplicate questionnaire at 4 weeks 


?e N6 



Figure 1 Design of Study. 



Twelve individuals were subsequendy deleted from the study when it was determined that they were out of 
the country or mail to them was returned by the post office as undeliverable. This left 276 individuals in the study. 



Pre!iminar>' letters were mailed cn April 3 io iliose individuals in groups receiving them (PI - P6). The 
preliminary letter emphas^ed the value of the graduate's participation and the importance of the study in improving 
the program for future students and in providing informaUon for the impending accreditaUon evaluaUon by NCATE. 

The quesUonnaire (an 8 page, 7" by 8 1/2" i vory<olored booklet) and a stamped, self-addressed return 
envelope were mailed one week later to all individuals. The accompanying cover letter for those who had not 
received the preliminary letter (Nl - N6) conveyed the same informaaon as the advance mailing. For those who had 
received the preliminary letter, the cover letter restated the value of the individuaVs participaQon. 

Reminders and subsequent mailings were sent only to those who had not responded. Those in the one-week 
reminder groups were mailed postcards (P3, N3) or leUers (P5, L5) one week following the mailing of the 
questionnaire. Those in the two-week reminder groups were mailed identical reminders after two weeks (P4, N4, P6, 
N6). A duplicate copy of the questionnaire and a cover letter (but no return envelope) wer ient two weeks after the 
reminder to all nonrespondents. This is referred to as the foUowup mailing. This was followed by a final reminder 
letter two weeks after the second questionnaire. 

For those in the groups not sent reminders, the duplicate questionnaires were sent three (Nl , PI) or four 
weeks (N2, P2) af ter the mailing of the original quesaonnaires, so that these individuals received their replacement 
questionnaires at the same time as those individuals who had been sent reminder postcards or letters. Two weeks 
after the duplicate questionnaire was sent, a final reminder letter was sent to nonrespondents. All mailings were sent 
first class (metered) except the postcards. 
Analysis 

Daily tabulations of returns for each condition were maintained. Because it would be impossible for a 
participant to receive a reminder and return the questionnaire in less than two days following mailing of the reminder, 
responses for each mailing were calculated for each condition by determining the number received through the day 
following the subsequent mailing. 

Cost per response per condition included the following costs: outgoing postage; return postage; duplication 
of letters, postcards, and quesaonnaires; mailing labels for envelopes; letterhead stationary and envelopes; and plain 
return envelopes. The costs used are presented in Table 1. Labor costs of designing questionnaires, hand signing of 
leuers, and stuffing envelopes were not included. The cost per return was detemined by dividing the total ( ost 
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involved for mailings in a condition (total costs calculated for the actual numbers of nonrespondents in the group 
for each mailing from preliminary letter through final reminder) by the final number of respondents in the gcoup. 

Table 1 
Base Costs 



Item 



Cost Per Unit 



Outgoing postage (university rate, first class, metered) for preliminary 

letters; questionnaires, cover letters & return envelopes; final follow-up letters $ .21 

Return postage ^ 25 

Stamps for postcards j 

Mailing labels $ 00176 

#9 Return Envelopes $ 0112 

Letterhead envelopes for outgoing mail 5 02395 

Letteriiead stationery $ 0215 

Duplication of single-page letters $ 0275 

Questionnaires (paper + duplication) 5 jj 

Duplication of postcards (message on plain card stock, then cut) $ .025 



Overall response rates and cost per response were calculated for each of the 12 conditions were calculated. (Because 
much of the previous research was conducted without benefit of followup mailings or with only a reminder after the 
initial mailing, rei>TX)nse rates and costs were also calculated up to the point of sending the followup that contained 
the duplicate questionnaire.) Chi-square analyses (p < .05) were used when observable differences in response rates 
between groups were potentially significant 

Results 

Res ponse Rate.5 

A total of 177 questionnaires were returned within two weeks of the final mailing for a return rate of 64%. 
Returns ^or the 12 subgroups ranged from 47.8% to 80.9%, as shown in Table 2. 
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Table 2 

Final Response Rate and Cost by Condition 



Condition 



Respon- 
dents 



Return 
Rate 



Cost per 
Return 



No preliminary letter 

Nl Duplicate at 3 w^ks 
N2 Duplicate at 4 weeks 
N3 Postcard reminder at 1 week 
N4 Postcard reminder at 2 weeks 
N5 Letter reminder at 1 week 
N6 Letter reminder at 2 weeks 

Ptk liminary letter 

PI Duplicate at 3 weeks 
P2 Duplicate at 4 weeks 
P3 Postcard reminder at 1 week 
P4 Postcard reminder at 2 weeks 
P5 Letter reminder at 1 week 
P6 Letter reminder at 2 weeks 

No preliminary leUer (N1-N6) 
Preliminary letter (P1-P6) 

Postcard reminder (N3, N4, P3, P4) 
Letter remin, r(N5,N6,P5,P6) 

One week reminder (N3, N5, P3, P5) 
Two week reminder (N4, N6, P4, P6) 



No reminder (Nl, N2, PI, P2) 
Reminder (N3-N6, P3-P6) 

No reminder: 

Duplicate at three weeks (Nl, PI) 
Duplicate at four weeks (N2,P2) 

Preliminary letter, no reminder (PI, P2) 
No preliminary, reminder (N3, N4, N5, N6) 
^X^ = 3.55, p = .0596 



24 


17 


/U.o /O 




23 


11 


47.8% 


$2.62 


24 


13 


54.2% 


$2.28 


23 


17 




CI CiC 

c>I.5o 


21 


17 


80.9% 


$2.01 


23 


14 


60.9% 


S2.31 


23 


IS 




CO 1/1 


24 


12 


:io% 


$2.80 


23 


iO 


78 'XCsL 




24 


15 


62.5% 


$2.33 


22 


11 


SO 0% 


s;'? 70 

»J> J. /u 


22 


17 


77.3% 


$2.11 


138 


89 


64.5% 


$1.97 


138 


88 


64.1% 


$2.41 


94 


63 


67.0% 


$1.99 


88 


59 


67.0% 


$2.43 


90 


59 


65.6% 


$2.36 


92 


63 


68.5% 


$2.06 


94 


55 


58.5% 


$2.16 


182 


122 


67.3% 


$2.20 


47 


32 


68.1%a 


S1.76 


47 


23 


48.9% 


S2.71 


47 


27 


57.4% 


$2.43 


91 


61 


67.0% 


$2.01 



The group with the highest response rate and the group with the lowest cost per return were both among 
those not sent preliminary leUers. The highest response rate (80.9%) was found from group N5, whose procedures 
included a letter reminder after one week. ITie lowest cost per return was $1.42 (group Nl), which resulted from 
sending the folJowup with a dupUcate questionnaire, after three weeks without benefit of an intermediate reminder. 

WhUe the first three paired comparisons showed little actual difference in response rate, there were 
noticeable differences in cost per response favoring omission of a preliminary letter, using postcards for reminders. 
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and sending reminders after fwo weeks. Sending a reminder of either type added litde to tiie cost per response but 
improved the response rate by 8.8%. When no reminder wdS sent, both cost and response rate were improved 
considerably by sending the foUcwup after three weeks rather than four. Finally, it was more effective and less 
cosdy to send a reminder than a preliminary letter. There were no statistically significant differences (p < .05) when 
chi-square comparisons were applied to response rates in the latter three pairs of conditions. Even a more liberal 
significance level (p < .10) had been used, only one condiUon would have shown significant impact. 

Discussion and Implications 
Results of this study indicate that when surveying a homogeneous group. s,uch as alumni, response rate is 
not improved by sending a jweliminary letter and serves only to increase the cost per response. The resources devoted 
to preUminary letters would be better utilized by sending a reminder. In general, postcard reminders were less costly 
but just as effecUve in inducing renims as letters. In deciding whether or not to send a reminder, the researcher must 
consider that the smaU increase in cost per return (when sending a reminder) produced a noticeable gain in response 
rate. A similar high response rate was attained at a noticeably lower cost per return by eliminating the reminder and 
sending a foUowup with a duplicate questionnaire after three weeks If reminders are used, sending them after two 
weeks was slightly more effecUve and less expensive than those sent after one week, as recommended by Cox 
(1966). A time lapse of one week after tlie first mailing, regardless of whether it be the advance latter or the 
quesUonnaire, does not always allow sufficient time for returns by the post office if the previous letter was 
undeliverable. This can result in multiple mailings to an obsolete address, adding to the cost per response. 

For economy, sending a second qaesUonnaire after three weeks without a previous reminder anc', without a 
preliminary letter would appear to provide the best method. If, however, return rate alone were the n'ain 
consideration, sending a letter reminder one week after the mailing of die questionnaire would be inserted into the 
model. The five approaches with the lowest cost were also the five approaches with the highest response rates, 
although the rank orders were not identical. Therefore, it would be at the discretion of die researcher to balance die 
two pieces of information and choose the most appropriate mcdiods. If a preliminary letter were used, sending a 
postcard reminder after one week would be die most advantageous in terms of boUi response rate and cost per return. 

Bodi response rate and cost must be considered when making decisions about procedures. The present study 
was limited because of the small sizes of the groups and because die survey was directed toward a homogeneous 
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population. In additic^., the researcher also imposed certain limiiations (uie number and iypcs of reminders and 
follovv.p mailings). Other researchers may end their efforts without sending reminders, without sending duplicate 
questionnaires, and/or without mailing an additional followup letter after the duplicate questionnaire. Any changes in 
basic procedures can alter the results. Variations in the focus of the study, the type of plea extended to the individual 
to respond, and the population being surveyed may yield diff^nt results even when the same timing and types of 
procedures described in this study are employed. Lack of statistical significance in findings should not cause the real 
or practical differences between comparisons to be overlodced. A difference of proximately 19% in response rate 
(favoring sending a followup after three rather than four weeks) would be tmporiant to a researcher even if that 
difference was not statistically significant Differences in costs per return, although not tested for statistical 
significance, must also be examined for practical significance to Uie researcher. 

The populations and the absence of subsequent followup mailings may account for differences between 
findings of the present study and previous research. Higher response rales were previously found when preliminary 
oradvanceconrespondence was used in surveys of consumers (Ford, 1967; Stafford, 1966; Walker & Burdick, 1977), 
car buyers (Heaton, 1965), or magazine subscribers (Eisinger, Janicki, Stevenson, & Thompson, 1574). These 
studies also examined response rates without benefit of followup mailings. 

/ Results of Uie present study are consistent, however, with those using similar populations. Parsons and 
Medford (1972), who surveyed what they called homogeneous populatiotis of alumni and religious leaders, found that 
advance notice :nade virtually no difference in iesponse rates of alumni, and provided conflicting results witii two 
groups of religious leaders. Kephart and Bressler (1958), also surveyed a homogeneous population, nurses. They 
were able to compare the benefits of a oreliminary letter with a followup mailing including a duDlicaie questionnaire 
and found Uiat the followup (with or witf.out a preliminary letter) produced a higher return rate tiian the preliminary 
letter witiiout a followup. The present findings also support the use of a remmder ratiier than a preliminary letter. 

Postcard or letter reminders have also been found to significantiy increase response rates of employees 
(Hinrichs, 1975) and the general public (Wiseman, 1973) when no further attempts were made to solicit responses. 
Moss (1981) also found a significant increase in response rate for university students due to a postcard reminder after 
one week when subsequent followups were sent Nichols and Meyer (1966) reported a significant increase in 
response rate attributed to three-day postcard reminders even with Uie mailing of a subsequent postcard reminder to 
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college students. That effect was negated by sending second wave quesiionnaif e& in a fullowup survey of former 
university students (Rossmaii and AsUn, 1974). The results of the present study are consistent with the findings of 
Moss and Nichols and Meyer that response rales were higher for reminder groups who sere subsequently sent another 
foUowup including a duplicate copy of the questionnaire and a final reminder. There is some support for Cox's 
suggestion that reminders be sent after one week, rather than two, but the other factors (type of reminder and whether 
or not a preliminary letter were rent) may also impact the effectiveness of this decision. 

To compare results prior to sending the followup and duplicate questionnaire, return rates and costs per 
return were calculated for all groups at ihe time at wh?ch the followup was sent (see Appendix). Sending a followup 
with a duplicate questionnaire end a final reminder negated the 10.5% difference (before the followup) in response rate 
favoring postcard reminders over letter reminders. The 2.9% advantage from sending a preliminary letter decrea*d to 
0.4%, and the 7.9% advantage of sending a reminder after two weeks (rather than after one week) was decreased to 
2.9%. The response rate for the one-week reminder group was measured prior to the followup (three weeks after the 
initial questionnaire was mailed).^There is a possibility that response rates might have been more comparable to the 
two week reminder group after another week (the time at which time the two week response rate was measured). 

Other response rate dif fererces. those between the three week and four- week groups that did not receive the 
reminder, between the rem:nder and no-reminder groups, and between those who received a reminder in place of a 
preliminary leUer. w-,re greater at the conclusion of the survey than prior to the followup mailing. This does 
reinforce the need to look at the total design when making comparisons between studies aiid to be cautious in 
attempting to generalize findings. Hopefully this study will be a first step in the development of a model for teacher 
education surveys and will provide the researcher with awareness of the available options and their benefits and costs. 
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Appendix 



Response Rate and Cost by Condition Prior to Sending a FoIIowup With a Replacement 

Questionnaire 



Condition 



No preliminary letter 

Nl Duplicate at 3 weeks 
N2 Duplicate at 4 weeks 
N3 Postcard reminder at 1 week 
N4 Toslcard reminder at 2 weeks 
N5 Letter reminder at 1 week 
N6 Letter reminder at 2 weeks 

Preliminary letter 

PI Duplicate at 3 weeks 
P2 Duplicate at 4 weeks 
P3 Postcard reminder at I week 
P4 Postcard remirder at 2 weeks 
P5 Letter reminr^ir at I week 
P6 Letter remi.ider at 2 weeks 

No preliminary letter (N1-N6) 
Preliminary letter (P1-P6) 

Postcard reminder (N3, N4, P3, P4) 
Letter reminder (N5, N6, P5, P6) 

One week reminder (N3, N5, P3, P5) 
Two week reminder (N4, N6, P4, ?6) 

No reminder (Nl, N2, PI, P2) 
Reminder (N3-N6, P3-P6) 

No reminder 

Duplicate at diree weeks (Nl, PI) 
Duplicate at four weeks (N2,P2) 

Preliminary letter, no reminder (Pi, P2) 
No preliminary, reminder (N3, N4, N5, N 6) 
^X^ = 3.03,p = .0816 



Rcspon- Return Cost per 
dents Rate Return 



24 


10 


41.7% 


51.59 


23 


7 


30.4% 


$2.27 


24 


8 


33.3% 


S2.45 


23 


12 


52.2% 


$1.62 


21 


7 


33.3% 


$3.15 


23 


7 


30.4% 


S3.19 



23 


10 


43.5% 


$2.29 


24 


5 


20.8% 


S4.59 


23 


12 


52.2% 


$2.25 


24 


11 


45.8% 


$2.36 


22 


6 


27.3% 


$4.89 


22 


11 


50.0% 


$2.64 


138 


51 


37.0% 


$2.29 


138 


55 


39.9% 


S2.88 


94 


43 


45.7% 


$2.19 


88 


31 


35.2% 


$3.34 


90 


33 


36.7% 


$2.99 


92 


41 


44.6% 


S2.42 


94 


32 


34.0% 


S3.53 


182 


74 


40.7% 


$2.35 


47 


20 


42.3%a 


$1.94 


47 


12 


25.5% 


$3.24 


47 


15 


31.9% 


$3.05 


91 


34 


37.4% 


$2.50 
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